Transforming neutral visual speech into expressive visual speech

We present a method for transforming neutral visual speech sequences into realistic expressive visual speech sequences. By applying Independent Component Analysis (ICA) to visual features extracted from time aligned neutral and equivalent expressive sequences, a model that separates speech from expression can be learned. Analyzing the behavior of different speaking styles in terms of this model provides both a means for identifying the component(s) responsible for expression, and for learning the correspondence between different speaking styles. Exploiting this correspondence to transform neutral visual speech into expressive visual speech creates sequences that have the same time varying expressive dynamics as the equivalent ground-truth sequences, and an objective analysis shows that the neutral ICA parameters are shifted into the appropriate ranges for expressive visual speech.